Gradient - Based Optimization of Markov Reward Processes :

A State Aggregation Approach to Singularly Perturbed Markov Reward Processes

In this paper, we propose a single sample path based algorithm with state aggregation to optimize the average rewards of singularly perturbed Markov reward processes (SPMRPs) with a large scale state spaces. It is assumed that such a reward process depend on a set of parameters. Differing from the other kinds of Markov chain, SPMRPs have their own hierarchical structure. Based on this special s...

Two Time-scale Gradient Approximation Algorithm for Adaptive Markov Reward Processes

In this paper, we study the stochastic optimization problem of adaptive Markov reward processes parameterized by two sets of parameters, including adjustable parameters, and unknown constant parameters. As the existing algorithms do not work well for this problem, we propose a novel two time-scale gradient approximation algorithm. This new algorithm yields fast convergence, small sample path va...

Localization and a Distributed Local Optimal Solution Algorithm for a Class of Multi-Agent Markov Decision Processes

We consider discrete-time factorial Markov Decision Processes (MDPs) in multiple decision-makers environment for infinite horizon average reward criterion with a general joint reward structure but a factorial joint state transition structure. We introduce the “localization” concept that a global MDP is localized for each agent such that each agent needs to consider a local MDP defined only with...

Asymptotic Behavior of Multivariate Reward Processes with Nonlinear Reward Functions

Approximate Gradient Methods in Policy-Space Optimization of Markov Reward Processes

We consider a discrete time, ®nite state Markov reward process that depends on a set of parameters. We start with a brief review of (stochastic) gradient descent methods that tune the parameters in order to optimize the average reward, using a single (possibly simulated) sample path of the process of interest. The resulting algorithms can be implemented online, and have the property that the gr...